High-speed image processing is required to operate robots at a high speed in a factory automation (FA) system or the like. For example, when a robot is configured with a feedback loop between its vision sensor and its actuator, the actuator can be controlled in units of milliseconds. Accordingly, the image processing speed that matches this speed is required. However, because the image processing speed in modern vision systems is limited by the video frame rate, the robot can only operate at low speeds that match this image processing speed. It is impossible to take full advantage of the robot's performance. Inspection systems with using vision sensors is limited in their accuracy and speed because of the restrictions of the video frame rate.
There are some high-speed CCD cameras that can take images within about one (1) millisecond. In these devices, the images are temporarily stored in memory and later read and processed. Therefore, the devices can be applied to such applications as image analysis. However, the devices have almost no real-time capability, and are not suitable for controlling robots, on-line inspections, or other applications.
In order to overcome these problems, institutes such as the Massachusetts Institute of Technology, the California Institute of Technology, and Mitsubishi Electric Corporation have been researching a vision chip that combines the image reading portion and the processing portion into one unit (“An Object Position and Orientation IC with Embedded Imager,” David L. Standley, Solid State Circuits, Vol. 26, No. 12, December 1991, pp. 1853–1859, IEEE); “Computing Motion Using Analog and Binary Resistive Networks,” James Hutchinson, et al., Computer, Vol. 21, March 1988, pp. 52–64, IEEE); and “Artificial Retinas—fast versatile image processors,” Kazuo Kyuma, et al., Nature, Vol. 372, Nov. 10, 1994). However, these chips employ a fixed analog circuit that is easy to integrate. Accordingly, these circuits have various shortcomings. That is, the circuits require subsequent-processing of output signals. The circuits suffer from the problems specific to analog signals. For example, the circuits are required to cope with noise and to attain a desired signal-to-noise ratio. The type of image processing they can perform is limited to special applications. The circuits have a lack of universality because the contents of the image processing are limited to particular applications.
To overcome these problems, a technology for a vision sensor capable of performing general purpose high-speed image processing is disclosed in Japanese examined patent application publication no. HEI-7-62866 (hereinafter referred to as conventional technology no. 1). For further improving integration, a technology for optimizing transfer lines in each row has been proposed in Japanese unexamined patent application publication HEI-10-145680 (hereinafter referred to as conventional technology no. 2). These types of vision sensor increase processing speed by providing a plurality of processing elements in one to one correspondence with a plurality of photodetector elements to achieve complete parallel processing. A technology for constructing a high-resolution sensor through partial parallel processing by providing one processing element for each array of photodetector elements is disclosed in International Publication WO95/24793 (hereinafter referred to as conventional technology no. 3). When used in inspection equipment on a production line, this type of vision sensor can greatly reduce the length of testing time.
In the actual use of these sensors, however, it is frequently necessary for a human being to visually confirm images while the images are being processed. When examining products on a belt conveyor, for example, a human worker is often entrusted with the final check. Hence, the image must be outputted to a display device that the worker can view. When setting and adjusting the equipment, in order to grasp how the inspection is achieved, visual confirmation is also necessary for checking lighting conditions on the image pick up position, lens focus, stains on the lenses, and other conditions. An efficient method of establishing processing algorithms for inspection methods is to set threshold levels and matching points while monitoring the processes by viewing actual images. It is possible to efficiently estimate the processing algorithms.
However, devices conforming to conventional technologies nos. 1 through 3 do not have an efficient function for outputting acquired images to external devices. These devices can only output results of calculations, in the form of characteristic quantities (such as the object's center of gravity and inspection results) or in the form of control signals (for controlling the motor and the like). Even if those devices are modified to output the acquired images, the frame rate of the images outputted from the high-speed camera is extremely fast and cannot be supported by a normal television monitor. Accordingly, a special display device is required to output the images, thereby increasing the cost of the system. Further, even if the images outputted from the high-speed camera can be displayed as they are, necessary details of the image may not be perceived by the human eye, which has a limited response rate (about 30 milliseconds).
A device for monitoring fast moving objects at a high-resolution has been disclosed in Japanese unexamined patent application publication HEI-10-21407 (hereinafter referred to as conventional technology no. 4). This monitor operates by overlaying image data from the high-speed image-processing device on image data from a conventional image-processing device. However, the frame rate of images outputted from the monitor of conventional technology no. 4 is limited to nearly the same video rate as the conventional rate. Therefore, it is difficult to extract the required information when observing fast moving objects.
Sometimes it is important to capture images at a particular timing, such as the instant of a press or the penetration of a drill performed by automated machine tools, the instant a part is inserted, the moment a golf club or a baseball bat meets a ball, and the like. It is difficult to display such images at the appropriate timing and in a manner that can be perceived by the human eye.